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1 Introduction 

Analysis of the wave propagation in waveguides with nonhomogeneous filling and 
arbitrary inclusions (perfectly conducting and dielectric) constitutes an important 
class of vector electromagnetic problems. However, many urgent tasks here remain 
unsolved that have been known for empty waveguides since the late 1940s; namely: 
existence of normal waves and their basic properties including the discreteness and 
localization the spectrum of normal waves on the complex plane, completeness and 
basis property in terms of both longitudinal and transversal field components, and 
so on. 

The theory of electromagnetic wave propagation in waveguides with homogeneous 
filling were elaborated in classical works of A.N. Tikhonov and A. A. Samarskii [40J- 
|12] . The analysis in this case is reduced to two scalar self adjoint problems which 
are studied using standard methods. List the most important results obtained for 
homogeneous waveguides: there exists a (countable) set of eigenvalues (spectrum 
of normal waves of a waveguide) consisting of real isolated points with the only 
accumulation point at infinity and the system of normal waves is complete and 
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forms a basis. For nonhomogeneous waveguides with given cross-sectional geometry, 
in particular, rectangular [521 [7] and circular [29] , the results concerning existence 
and distribution of the normal wave spectra on the complex plane are obtained by 
reducing to explicit dispersion relations and analysis of the corresponding complex- 
valued functions of one or several complex variables. 

To the best of our knowledge, the existence of eigenvalues and their distribution 
on the complex plane remain an open issue as well as the completeness and basis 
property for the system of normal waves in nonhomogeneously filled waveguides with 
arbitrary inclusions. This fact has become a main reason for us to complete in this 
paper the mathematical theory of wave propagation in waveguides by filling these 
gaps. 

Let us briefly summarize the new fundamental results obtained in this study for 
an arbitrary waveguide with nonhomogeneous filling and arbitrary inclusions (that 
belongs to the considered family): 

(i) the spectrum of normal waves is nonempty and forms a countable set of isolated 
points on the complex plane (cut along two intervals on the real axis) without finite 
accumulation points; 

(ii) the spectrum is symmetric with respect to the axes on the complex plane, is 
localized in a strip, and contains not more than a finite number of real points. 

During the last two decades an increasing interest has been reported to the study 
of electromagnetic wave propagation in guiding systems with nonhomogeneous filling. 
Different types of them have been created and found various practical applications 
and many their physical properties have been established. Simultaneously, the in- 
terest in developing rigorous mathematical techniques has never vanished. A driving 
force here is the necessity of designing new guiding systems such as complicated 
volume and planar microstrip and slot transmission lines where nonhomogeneous 
structure of the guide plays the crucial role. Note also that the study of the wave 
propagation in waveguides with inhomogeneous filling requires (and leads to) elabo- 
ration of special methods of the spectral theory of operator- valued functions (OVFs) 
and operator pencils. 

The typical settings that arise in mathematical models of the wave propagation in 
nonhomogeneous waveguides are nonself adjoint boundary eigenvalue problems for the 
system of Helmholtz equations with piecewise constant coefficients. On the medium 
discontinuity lines (or surfaces) the transmission conditions are added. An important 
feature is that the spectral parameter enters both the equations and transmission 
conditions in a nonlinear manner. A huge amount of publications is devoted to 
investigations of these problems. However, the main attention was paid to numerical 
determination of dominant modes propagating in waveguides of various structure; 
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many references can be found in [5l EJ [54j [HI EU [43] . 

Analysis of the propagation of normal waves in waveguides with nonhomogeneous 
filling is reduced to a vector nonself adjoint boundary value problem. Complex waves 
may exist in such waveguides that correspond to eigenvalues which are neither purely 
real nor purely imaginary. This phenomenon was discovered and studied in [31 [51] . 
The existence of eigenvalues of multiplicity greater than 1 was discussed in [15j [26] . 

Important contribution to the mathematical theory of electromagnetic wave prop- 
agation in waveguides of complicated structure was made by A.S. Ilinski and Yu.V. 
Shestopalov in [12] [H] and [13] [32]. They propose the reduction of the problem on 
normal waves in a waveguide to a problem on characteristic numbers for a meromor- 
phic OVF nonlinear with respect to the spectral parameter; in the majority of cases 
OVF is an operator of a system of integral equations with logarithmic singularity 
of the kernel. This approach was developed also by E.V. Chernokozhin [TT] and in 
[16]-|20j. Using this technique, the discreteness of the spectrum of normal waves 
was proved for a wide family of waveguides with nonhomogeneous filling. For slot 
transmission lines the existence of eigenvalues was established in [13] . Localization 
of eigenvalues on the complex plane were studied in [T81 [TTJ [T31 EE3] • Note however 
that it is hardly possible to prove the existence and determine the spectrum location 
on the complex plane by these methods for a wide family of nonhomogeneously filled 
waveguides considered in this study. 

An approach based on the reduction to eigenvalue problems for operator pencils 
considered in Sobolev spaces was proposed by Yu.G. Smirnov in [471 EH [49] . General 
theory of polynomial operator-functions called operator pencils is sufficiently well 
elaborated in [21 EJ El EE ESJ [30] and [33]- [35]. A fundamental work by Keldysh [22] 
pioneered investigation of nonself adjoint polynomial pencils. Note that the theory 
of operator pencils is very close to the theory of nonself adjoint operators [21 [9] and 
allows one to apply powerful methods of the latter. Operator pencils were applied 
to the analysis of electromagnetic problems in [SI E51 12"T] . 

We see that the method of operator pencils has proved to be a natural and efficient 
approach for investigation of the wave propagation in waveguides. The reduction of 
boundary eigenvalue problems to eigenvalue problems for operator pencils allows one 
to apply various well-developed methods of functional analysis [39] in order to study 
spectral properties of the pencil. This method is applied in the present study. 

Let us give a brief insight into the contents of this work. In Section [21 we describe a 
class of waveguides under consideration and formulate the problem on normal waves 
for homogeneous Maxwell equations stated in terms of longitudinal components of 
electromagnetic field. We perform the reduction to a boundary eigenvalue problem 
for the system of Helmholtz equations and introduce the notion of (weak) solution 
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using variational relations in Sobolev spaces. Among characteristic features of the 
problem note that the spectral parameter enters the transmission conditions in non- 
linear manner, waveguides are filled with nonhomogeneous media, and the boundary 
has 'edges'. Therefore, a special definition of the solution is required. We formulate 
this definition using variational relations. 

In Sections [3] and H] the problem is reduced to the study of an operator pencil 
L (7) of the fourth order. We investigate the properties of the operators of the pencil 
and establish basic properties of its spectrum showing among all that the pencil does 
not belong to the families of Keldysh pencils or hyperbolic pencils. Finally we prove 
fundamental theorems concerning the discreteness and localization of the spectrum 
on the complex plane. 

The techniques used in this study are mainly based on the approaches and results 
employing the methods of nonself adjoint OVFs and operator pencils published in 

unmans]. 

2 Statement of the problem on normal waves in a 
waveguide 

Let Q C R 2 = {x 3 = 0} be a bounded domain on the plane Ox\X 2 with boundary 
dQ. Let 1 C Q be a simple closed or unclosed C°°-smooth curve without points of 
intersection, dividing Q into domains Qi and £1 2 ; Q — ^1 U U 1. If 1 is an unclosed 
curve, then the points d\ do not coincide and belong to dQ: dl C dQ. We will assume 
also that boundaries dQ, dQi, and dfl 2 of domains Q, Q\, and Q 2 are simple closed 
piecewise smooth curves formed by a finite number of C°°-smooth arcs intersecting 
at nonvanishing angles. 



Figure 1: Geometry of the waveguide cross section of the first type. r = T' U dQ. 

r u r u Pi = 1 (r n r = 0), n = n t u n 2 u r. 
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Let pi G 1 be arbitrary 2iV points pi ^ pj dividing 1 into parts T and T' such that 

r = i\F, r = i\r, r u r up* = 1 (r n r = 0). if n = o then r = 1, r = 0. We win 

i 

also use the notation Q = Q± U S7 2 U T and T = dQ U V. 

In the general case boundary dQ of domain Q contains the points with inner 
angles < a < 2tt. If a = 2n, such a point is called edge. Domain Q satisfies the 
cone property which allows us to apply the embedding and trace theorems in Sobolev 
spaces [U [37] . 



dQ 




Figure 2: Geometry of the waveguide cross section of the second type. 



We will consider the problem on normal waves in a cylindrical shielded waveguide 
whose transversal (with respect to OX3 ) cross- section is formed by domain Q. We 
will assume also that waveguide's filling contains two isotropic media with relative 
permittivity Sj in domain Qj; Sj > 1, ImEj = 0, and pj — 1 (j = 1,2). Here T is 
the projection of the surface of the infinitely thin and perfectly conducting shields 
and T is the projection of the dielectric surfaces. 

This family of waveguides contains in particular all types of shielded transmission 
lines: cylindrical and rectangular waveguides with partial filling, slot lines and strip 
lines with several slots or strips placed on a curved interface etc. [51] . 

Propagation of electromagnetic waves in a guiding system is described by the 
homogeneous system of Maxwell equations with dependence e nX3 on longitudinal 
coordinate [14] : 

rotE = — iH, X = (xi, X2, X3) G E, 
rotH = ie~E, x = (xi, x 2 ) , 

E (X) = {E 1 (x) e x + E 2 (x) e 2 + E 3 (x) e 3 ) , 1 ' 

H (X) = (H, (x) ei + H 2 (x) e 2 + H 3 (x) e 3 ) e^, 
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with the boundary conditions for the tangential electric field components on the 
perfectly conducting surfaces 

E t | M = 0, (2) 

the transmission conditions for the tangential electric and magnetic field components 
on the interface (surfaces where the permittivity is discontinuous) 

[E t ] L = 0, [H,] x = 0, (3) 

and the condition that provides finiteness of energy 

J (e|E| 2 + |H| 2 )dX < oo. (4) 

v 

Here M = {X : x G To} is the shielded part of the boundary, L = {X : x G T} is 
the boundary where the permittivity undergoes breaks, and V C £ = {X 
is an arbitrary bounded domain. System of Maxwell equations ([T]) is written in the 
normalized form and we use the following dimensionless variables and parameters 
[5^1 I2"0] : k x — > x, a/ /Uo/^oH — » H, E — > E; k\ = e fi u 2 , where e and fi are 
permittivity and permeability of vacuum (the time factor e luJt is omitted). 

The problem on normal waves is an eigenvalue problem for the system of Maxwell 
equations with respect to spectral parameter 7. Eigenfunctions corresponding to 
certain complex values of a longitudinal wave number 7 are usually called the normal 
waves of the waveguide. 

Write system of Maxwell equations ([1]) in the form 

dH 3 dE 3 . dH 3 . 

— i'yH 2 = ieE 1 , i^E 2 = -iH 1 , zjHt-— — = ieE 2 , 

OX 2 OX 2 OX\ 

. dE 3 8H 2 dHx . „ dE 2 dE x 

ijEx - — = -%H 2 , — = ieE 3 , — — = -iH 3 , 

axx ox 1 ox 2 oxx ox 2 

and express functions Ex, Hi, E 2 , and H 2 via E 3 and H 3 from the first, second, 
fourth, and fifth equalities 

i ( dE 3 dH 3 \ ^ i ( dE 3 dH 3 \ 

i f dE 3 dH 3 \ i ( dE 3 dH 3 \ p 

Note that this representation is possible if 7 2 7^ E\ and 7 2 7^ e 2 . 
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It follows from (jSJ) that the field of a normal wave can be expressed via two scalar 
functions 

II (xi,x 2 ) = E 3 (xx,x 2 ) , * (xx,x 2 ) = H 3 (xi, x 2 ) . 

Thus the problem on normal waves is reduced to a boundary eigenvalue problem 
for functions II and Let us write down this problem. From ([1]) and ([2]) we have 
the following eigenvalue problem: to find 7 G C called eigenvalues such that there 
exist nontrivial solutions of the Helmholtz equations 



An + k 2 Tl = 0, 



x 



(xi,x 2 ) G Qi U Q 2 , 



+ = 0, k 2 
satisfying the boundary conditions on r 



*5 



7 



ni 



r 



0. 



dn 



(6) 



(7) 



the transmission conditions on T 



[U] r = 0, [*] r = 
and the energy ('edge') condition 





" 1 




" e dW 


= 0, 

r 




" 1 dW 




" 1 


7 




+ 

r 
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r 





J (|vn| 2 + |w| 2 + |n| 2 + \^\ 2 )dx < 00. 



0. 

(8) 
(9) 



Here n and r denote the (exterior to Q 2 ) normal and tangential unit vectors such 
that x\ x x 2 = r x n. Square brackets [/] r = f 2 \ r — fi\ r denote the difference of 
limiting values of a function on T in domains ^2 and Conditions (J7]) are to be 
satisfied on both sides of the part of boundary T'. 

In order to obtain we used formulas (j3J). Conditions ©-Q are another 

form of conditions ©-(jl]). Thus the longitudinal components of a normal wave 
satisfy ©-([9]). The inverse assertion is true. If IT and ^ is a solution of problem 
dn]H© then the transversal components can be determined by (JS]). The field E, 
H will satisfy all conditions ([[]) and ©-(jl]). The equivalence of the reduction to 
problem is not valid only for 7 2 = e\ or 7 2 = e 2 ; in this case it is necessary 

to study system p]) directly. 

System of equations © with boundary conditions (JTj), transmission conditions 
(|S]), and condition (P) constitutes a boundary eigenvalue problem that will be a 
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subject of our study. Note that coefficient e is not continuous and the transmission 
conditions contain spectral parameter 7. Moreover, boundary V may have 'edges'. 

Let us formulate a definition of the solution to problem (IHJ) ([9j) that will be used 
in the further analysis. 

We will look for solutions to problem (JS}-© in the Sobolev spaces [TJ [37] 

Hi (fl) = {/ : / G H 1 (fl) , /lr = 0} , H (fl) = j/ : / G H 1 (fl) , | /dr = J 

with the inner product and the norm 

(/,<?)!= J VfVgdx, H/li; = (/,/)!■ 
h 

^ 1 

The seminorm || • || x in if 1 (Q) is a norm in Hi (fl) and H (fl) because sesquilinear 
form (f,g) 1 in these spaces is coercive pp. Note that it is sufficient to use the 
boundedness of fl in order to prove that the form is coercive in Hi (fl); however it 
is necessary to use the cone property for the proof of the coercive property of the 

form in H (fl)). Spaces Hi (fl) and H (fl) can be defined as a supplement of spaces 

of infinitely smooth functions (fl) and C°° (fl) with respect to the norm || • || x 

_ 1 

(under the condition \\fWi < 00); H (fl) is a subspace of functions from H 1 (fl) 
which are orthogonal to the unit function. 

Under the above assumptions the domain fl satisfies cone property: there is a 
cone 

K Q = {x : < xi < b, x\< ax\\ a > 0, b > 0} 

such that any point P G fl can be a vertex of cone K p which is equal to K , and 
the cone belongs to fl, K p C fl. This property allows us to apply the Sobolev trace 
theorem [TJ and consider the trace of function / e H 1 (fl) on T as an element of space 
H x l 2 (r). Due to the trace theorem, the relation /| r = means that the function 
is equal to zero in if 1//2 (To). For any function / G H 1 (fl) we have [/] r = in the 
sense of space H 1 / 2 (T); and vice versa, if [/] r = 0, f\ u G H 1 (fli), f\ u G H 1 (f2 2 ), 
then / G H 1 (fl) . On the part of the boundary T' C r the trace theorem should 
be applied on both sides of T'; in this case functions / G H 1 (fl) have in general 
different traces on different sides of T'. Note also that the following embeddings 

tfo 1 (fl) C Hi (Q) C H 1 (Q) C H 1 (fl) , 
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hold but all embeddings are not dense if V ^ 0. 

„ 1 

Assume that II G Hq (f2), ^ E H (Q). Condition is fulfilled in fii and f2 2 in 
terms of distributions [31]. Moreover, we have for the boundary condition on Tq 



n 



-jlr n9o. 



e H 1 ' 2 (r„ n dfy 



<9n 



e #~ 1/2 (r n 5%) . 



r nan, 



For the transmission condition on T, we have 

n| r g # 1/2 (r) , ^| r G# 1/2 (r) 



<9n 



<9^ 



<9n 



^- I/2 (r). f 



a7 



g h- 1/2 (r) 



where H, and are restrictions of II and \1/ on fly. 

Let us give a variational formulation of problem (J3j) — (J9l) - Multiply equations ([3]) 

and (J4]) by arbitrary test functions u G Hq (Q) and v E H (Q) (we may assume that 
these functions are continuously differentiate in Cli and Q2 because these spaces 

form dense sets in Hq (Q) and H (fi)), and apply Green's formula [31], [I] for each 
domain Qj separately. Note that the possibility of applying Green's formula for these 
functions is proved in [31] and [I], p. 618. We have 



VllVudx - k 2 / Uudx 



V^Vvdx - k) 



on 

dn 



^vdx 



dn 



udr, 



an. 



an, 



vdr; j = 1,2. 



(10) 



11) 



an, 



Then, substituting the normal derivatives from ([7]) and ([8]) to (TTOT) and (TTTT) we obtain 
the variational relation 



yi(£VIIVM + VWu)dx- J (sUu + ^v)dx-^y i ^ 



/<9n 



- 



7fr 



u ) dr = 0, 



(12) 



which is derived for smooth functions u, v. In Section 2.1 we will prove the continuity 

of the sesquilinear forms defined by the integrals in ( ITS]) . Hence relation ( !T2|) can 

„ 1 

be extended to arbitrary functions u G Hq (f2), v G H (Q). Here and below, the / 
under the integral sign in J fdr is the trace of the function on Y. 



For v = 1, u = we obtain in a similar manner 



*(/./• = - 
n r 



1 



dr = —7 



1 



dli 



dr = 0: 



(13) 



consequently, the choice of space H (fl) does not contradict to the choice of the 
space of solutions to problem (jSJ)-©. In (jl"3l) we used the condition 



% e if 1/2 (r) := {<p : <p e i/ 1/2 (i), su PP <p c r}, 

since II e -fig 0^)> so that the set of functions Cq° (T) is dense in if 1 / 2 (f ). 
Definition 1. The pair of functions 

UeH^(Q),^eH(Q) (\\U\\i + ll*lli^0) 
is called the eigenvector of problem (G|)-(GJ) corresponding to eigenvalue 70 if varia- 

tional relation [Wi) holds for u G iio(f2), v E H (Q). 

_ 1 

Thus, if IT G i^o 1 (fi) and ^ E H (fi) and ©-Q are fulfilled, variational relation 
(j!2p also holds. The inverse assertion is true. Choosing u and v with a support in Qj 
we have that equations ([6]) are fulfilled in terms of distributions. The first condition 

in (J7J), the first condition in (jSJ), and ([9]) are fulfilled by the definition of spaces Hi {ft) 

„ 1 

and if (fi). If we choose u = and assume that the support of v contains the part 
r x of boundary r , then from ([T2"j) and Green's formula we find [1], [21] that the 
second condition in (jTJ) is also fulfilled in term of distributions. Choosing arbitrary 
u and v on V in ([121) and applying formulas (flOl) and (flT!) we obtain the relation 




1 

¥ ~dr 



+ 



J r 



7 



1 <9n 

p^ 7 



1 dV 



ug?t = 0, 



are fulfilled in terms of 



which yields that the second and third condition in 
distributions. 

Let us give some remarks concerning the smoothness of eigenvectors of problem 
(JSJ) — (JSJ) - It is well known [28| [38] that solutions II and \l/ of homogeneous Helmholtz 
equations (jSJ) are infinitely smooth in fli and f^: II, ^ G C°° (Oi U Q2)', consequently, 
equations (EI) are satisfied in the classical sense. In a vicinity of an arbitrary smooth 
part r x of boundary T conditions flTJ are also fulfilled in the classical sense and 
functions II and \I/ are infinitely smooth up to the boundary. The behavior of II and 
^ close to edge (angle) points was analyzed in [21]. Note that in what follows we 
will not use the smoothness of functions II and 
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3 Eigenvalue problem for operator pencil 

Multiplying (JX2J by k\k\ we rewrite it in the form 

7 4 / (ellu + dx + 7 2 ( / (eVTIVw + V* Vt>) cb- 

n / r (14) 

+6^2 I J (eUu + *v) tte - / (VnVw + ±WW) rfx J =0, 

Vw e Hq 1 (O) , veH (fi) . 

„ i 

Let = ifg (fi) x if (O) be the Cartesian product of the Hilbert spaces with 
the inner product and the norm 

(f,g) = (h,9i) 1 + (f2,g2) 1 , ||/|| a = ||/i|i; + ||/2|i;, 

where 

f,gEH, f=(f 1 ,f 2 f, g = ( gi ,g 2 f, f^g.eH^n), f 2 , g 2 e H (Q) . 

Then the integrals in ( 1141) can be considered as sesquilinear forms on C defined in 
H with respect to vector-functions such that 

fi = n, f 2 = 5-i = tt, 5-2 = u. 

These forms (if they are bounded) define, in accordance with the results of [21], linear 
bounded operators T : H — >• H 

t(f,g) = (Tf,g), VgeH. (15) 

Linearity follows here from the linearity of the form with respect to the first argument 
and continuity from the estimates 

||T/|| 2 =t(/,T/)<C||/|| ||T/||. 

Consider the following quadratic forms and corresponding operators 

at (/, g) := J (eVftfgx + V f 2 Vg 2 ) dx = (A 1 f, g) , Wg e H, 
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J fafiVgi + ^V/ 2 Vg 2 ) dx = (A 2 f, g), Wg G H, 



«2 {f,g) ■-- 



k (f, g) := J (e/^ a + f 2 g 2 ) dx = (Kf, g) , Wg G H, 
h 

s (/ ' ^ := / (l^ 2 ~ §7^) dr = {SL g) ^ ge H - (16) 

r 

It is easy to see that forms a\ (/, g) and a 2 (/, g) are bounded. The same property 
for the form k (f,g) follows from Poincare's inequality [TJ. 

Let us prove that form s (/, g) is also bounded. Assume that functions fx, f 2 , gi, g 2 G 
C 1 (fi a ) nC 1 (fi a ). Then 

dfx _ d h- \ , [if dfi dg 2 dfx dg 2 df 2 dgx df 2 dgx \ , 

-dT 92 -^ 91 ) dT = J 2W 2 ^x~d^xd^ + dx-xdx-2~dx-2dx-J ^ 

u 

where 

^ { 1, x e Qx 



— 1, I G (] 2 

Using the Schwartz inequality we finally obtain 

\s(f,g)\< \\\f\\ \\g\\. (17) 

The required property of the form may be easily obtained if to extend the estimate 
given by (TTT|) for arbitrary functions f,gEH using the continuity. 
Remark. In the expression 

dfx- df 2 _\ 
gii 92, q^i and ^ mean the restriction of elements 

G H- 1 ' 2 (dQj) 



I r- trl/2 /an \ ^/l ^ 



on T [2TJ. Since on 1 

/ilr' = > 9i\r> = °i supp/iCf, supp #i C f , 
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the following formulas of the integration by parts hold 

/f^ T = -/t^ T ' I^~ 9ldT= -J^ f2dT - (18) 
r r r r 

Now the variational problem given by ( IT4|) can be written in the operator form 

(L( 1 )f,g)=0, VgeH, 

which is equivalent to the following equation for the operator-valued pencil 

L(7)/ = 0, L^-.H^H, 
L ( 7 ) := 7 4 K + 7 2 (A 1 - ( £l + e 2 ) K) + ( £l - e 2 ) + e x e 2 (K - A 2 ) , 

where all the operators are bounded. 

Equation f|T9|) is another form of variational relation (j!4p . Eigenvalues and eigen- 
vectors of the pencil coincide with eigenvalues and eigenfunctions of problem ([H])-® 
for 7 2 7^ Ei, 7 2 7^ e 2 according to the definition. 

Thus the problem on normal waves is reduced to an eigenvalue problem for pencil 

Hi)- 

Now we consider the properties of the operators in ( !T6|) . 
Lemma 1. The operators A\ and A 2 are uniformly positive: 

I<M< e ma J, e~iJ <A 2 <I, (20) 

where £ max — max {e\, e 2 ) and I is the unit operator in H . 

The proof of the lemma is reduced to the verification of simple inequalities 

ll/ll 2 < (A x f, f) < e max Il/H 2 , e m l ll/ll 2 < (A 2 f, f)<\\ff. 

Lemma 2. The operator S is selfadjoint, S = S* , and the following inequalities 
hold 

-\l<S<\l. (21) 
The selfadjointness of operator S follows from the equality 



dfx- d h - \ , f ( dgi dg 2 \ 



which follows in its turn from the remark above and formulas (|T8|) . Inequalities (12 ip 
follow from fj!7j) . 
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Lemma 3. Operator K is positive, K > 0, and compact. The following estimate 
holds for its eigenvalues 



X n (K) = O (n" 1 ) , n -»■ oo. (22) 



It is easy to see that (Kf, /) > for / ^ since J (e \ fi\ 2 + I/2I 2 ) dx = is 

n 

fulfilled only for f 1 = and f 2 = (in tf 1 (fi)). 

The compactness of the operator K follows from formula ( 1221) . 

The proof of formula (|22|) is based on the Courant variational principle. From 
the inequality 

j (^|/i| 2 + |/ 2 | 2 ) dx<e^J (l/^ + l/sl 2 )^ 
n n 

we obtain [8] that 

A n (If) < c max 

where A n (K^) are eigenvalues of the operator specified by the sesquilinear form 

q (f, 9) ■= j (fi9i + f29 2 ) dx = (K H f, g) , \/g £ H. (23) 
h 

Thus it is sufficient to consider operator Kh- Formula ( 1221) follows from the asymp- 
totic behavior of s-numbers [8] of operator K h which can be found in [50] (Theorem 
4.10.1). The statement is proved. (The proof in detail one can find in [4"6] ) 

Thus all operators Ai, A2, K, and S are selfadjoint, and KerK = {0}. There 
exist bounded inverse operators Aj : H — )■ H and A- , Aj : H — > i/; these 
operators are uniformly positive. Note that in this case the condition B > leads 
to B = B* since Hilbert space H is considered on complex-valued functions. Using 
these lemmas we obtain 

Corollary 1. Operator pencil £(7) is self-adjoint: 

V ( 7 ) = L (7) ■ (24) 

From variational relation ([141) we obtain 
Corollary 2. Lei P be an operator such that 



P(/i,/ 2 ) T = (-/i,/ 2 ) 



T 
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Then 

Ax = PA X P, A 2 = PA 2 P, K = PKP, S = -PSP, 
and the following representation holds 

L(- 1 )=PL{ 1 )P (25) 

The proof of this statement follows directly from the explicit form of variational 
relation (fill) . 

Note that operator S does not possess the Fredholm property because dim Ker S = 
oo. Indeed, all functions / = (/i,/2) T satisfying the additional conditions /i| r = 0, 
/ 2 | r = belong to the kernel of operator S. 



4 Properties of the spectrum of pencil L (7) 

We will denote by 1Z (L) the resolvent set of L (7) (consisting of all complex values of 
7 at which there exists a bounded inverse operator L~ l (7)) and by o (L) = C\1Z (L) 
the spectrum of L (7). 

In what follows we will use the definitions of finite-meromorphic OVFs and canon- 
ical system of eigenvectors and associated vectors of an OVF formulated in [8| [TOl 123] . 
We will consider OVFs that have eigenvalues with finite algebraic multiplicity. 

Let us study the spectrum of pencil £(7). It is more convenient to consider a 
regularized pencil 



L (7) := A~ 1/2 L (7) A' 1 ' 2 = + 7 2 (/ - (ei + e 2 ) K 



(26) 



where K = A~ 1/2 KA~ 1/2 , S = A~ 1/2 SA~ 1/2 , and A 2 = A~ 1/2 A 2 A~ 1/2 . 

It is easy to see that a (L) = a (l \ and the following relations hold for eigenvec- 
tors and associated vectors 

W (L) = A- 1/2 ^ (Z) . (27) 

Operators K, S and A 2 keep all properties of operators K, S, and A 2 given in 
Lemmas 1-3 with the estimates 

-\l<S<\l, e m U<A 2 <I. (28) 
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Properties of the spectrum of pencil L (7) are summarized in the following theo- 
rems. 

Theorem 1. 



a (L) c IL; = {7 : \Rej\ < 1} 

i. e. for a certain I > the spectrum of pencil L (7) lies in the strip 11/ . 

Proof. In order to prove this theorem, assume that I > y/e[ + e 2 and consider 
the operator-valued function 

F (7) '■— (7 2 - {ei + e 2 )) 1 L (7) = j 2 K + I + (7) , (29) 
in the domain D = {7 : |7| > I}, where 

T (7) = 7 ( 7 2 - ( £l + e 2 )) 1 x (( £l + e 2 ) 7 + ( £l - e 2 ) 7 S + e x e 2 (k - A 2 )) 

is a holomorphic and bounded operator- valued function in the domain D : \\T (7) || < 
T for 7 G -Do- One can see that a (jlj fl D = a (F) PI Do- If | -Re-yj > I, there exists 
a bounded operator 

R( 1 ) = ( 1 2 K+iy 1 

and its norm can be calculated by the formula (see [9], p. 309) 

ll«(7)| 



d(-r 



2V 



where <i(/i) is the distance from point \x to spectrum of operator K. Thus we obtain 
the estimates 

we mii < I 7 ' 2 ' = = 1 (hi + t^H < 1 A + M 

11 - |/m 7 - 2 | 2|7'|| 7 "| 2 \Jt"| M/ 2 V I 

when |7"| > |7'| and ||i?(7)|| = 1 for < |7'| where 7 = 7' + ij". Choosing the 
value / > T we obtain 

,-lrrfS D , mi 1 A , MA ^0 



117^(7)^(7)11 < 5 ^ + ^J^<1- 
Hence, there exists a bounded operator 

F- 1 ( 7 ) = it;(7) (/ + 7 -1 r (7) ^(t))" 1 , 
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which yields the existence of bounded operators L~ l (7) and L~ x (7) for 7 6 11/ = 
{7 : \Rej\ < 1} outside the strip 11/. 

Corollary 3. The resolvent set of pencil L (7) is not empty, 

c\n, cn(L). 

Theorem 2. The spectrum of pencil L (7) is symmetric with respect to the real 
and imaginary axes: 

a (L) = a(L) = -o (L) . 

If 7o is an eigenvalue of pencil L (7) corresponding to the eigenvector (II, \1/) T then 
— 70)7o; an d —70 are also eigenvalues of pencil L (7) corresponding to the eigenvectors 
(-n, , (n, ^) , and (-11, with the same multiplicity. 

Proof. The first assertion of Theorem H] follows from (EMI) and fTSo^) . Proof 
of the second assertion is actually a simple verification of variational relation ( TT4|) . 
Note that associated vectors at 70 can be obtained by taking complex conjugation 
of associated vectors corresponding to 70. 

Theorem 3. Set 5 = (e 2 - £1) /2, 

J = < 7 : ^7 = 0, < |7| < 

In the domain C\Iq the spectrum of pencil a (L) is a set of isolated eigenvalues with 
finite algebraic multiplicity. The points jj = iy 7 ^ (i = 1,2^ are the degeneration 
values of pencil L (7) : dimker L (7^) = 00. 

Proof. Taking 7 = 7' + i'-f" with 7" / we have 



1 (7^ - 25S - £ -^A 2 
7 V 7 



hence, in line with [9], the operator 



+ £ -^A 2 > I- 
7 



^0 (7) := 7 ^1 - 27^5 - £i£ 2 ^2 

has a bounded inverse and L (7) is a Fredholm pencil with ind L (7) = 0. 

Introduce the operator A[ : H — >• if defined by the form (£ was defined before 
formula (JTTj) ) 

ai (/, 9) ■= j £ {eVhVg x + Vf 2 Vg 2 ) dx = (A'J, g) , Vg G F, (30) 
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and the operator 



A[ = A~ 1/2 A[A- 1/2 . 



For these operators the following estimates hold 




I <A[< I. 



(31) 



Set p = ((e a + £1) /2) 1/2 . For real 7 £ J the estimate I7 2 - p 2 | > |5| (1 + (7!) holds; 
consequently, the operator 



has a bounded inverse as well as operator L (7) . L (7) is a Fredholm operator with 
zero index. Here, we used estimates (J2SJ) and f l3Tj) . 

The second assertion of the theorem follows from variational relation (|T4|) for 

7 = 7j and 



for Cl C Oi and Cl C fi 2 . 

From the physical viewpoint the real and pure imaginary points of spectrum a (L) 
are of interest because they correspond to propagating and decaying waves. It should 
be noted however that 'complex' waves may exist j3l[52] for 70 G o (L) and 7q-7o 7^ 
(70 = 7o + *7o)- I n the general case, strip II; in Theorem @] cannot be replaced by 
the set 



From Theorem H] it follows that complex waves occur in 'fours'. Note also that if a 
waveguide has a homogeneous filling {e.\ = e 2 ) then there are no complex waves. 

The existence of spectral points does not follow from Theorem H] (except for the 
points 7j = ±y^Fi). The proof of existence of a countable set of eigenvalues of L (7) 
with an accumulation point an infinity will be given below. Note that at the points 
7j = ±y/ei, the reduction of the boundary value problem on normal waves to the 
eigenvalue problem for the pencil is not valid (see Section 2.1). Hence one may expect 
the occurrence of eigenvectors of pencil L (7) corresponding to eigenvalues jj. Using 
the methods of potential theory [TJ] we can prove that L (7) is a Fredholm pencil for 
all other real points 7. Note that there are no other degeneration points and finite 
accumulation points. 

Let us prove the existence of discrete spectrum of pencil L (7). First we prove 
the following statement. 



L (7) := A- 1/2 L (7) A^ 12 = ( 7 2 - p 2 ) I - 2 7 55 + 6A\ 




n = {7 : (Rej) ■ (Imj) = 0} . 
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Lemma 4. If the vector-function (7) = F' 1 (7) (7 _1 /o + fi), fo, fi £ H , is 
holomorphic for \ j\ > Rq with a certain Rq > then this vector-function is uniformly 
bounded (with respect to the norm) on this domain. 

Let A be the angle 

A 

Then for all 7 ^ A we have the estimates (see the proof of Theorem H]) 





7T 




3tt 






arg7 - - 


<e, 


arg7 - y 


<,} 



. , , . < - (1 + cot I 

under the condition > |Y|. We also have ||i?(7)|| = 1 for |7'| > Thus 

11^(7)11 < l + cot0, A. (32) 

Assume that I7I > R\ > To (1 + cot #) (value To was defined in the proof of 
Theorem 1), R\ > Rq and 7 ^ A. Then the inequalities 



T „„, ,,A A 1 + cot# 



1^(7)11 < \\R(l)\\ (1-^11^(7)11) < 



1 - Q (1 + cot 0) 



(33) 



Ri 



follow from fl32|) . Moreover, if the vector-function </3 (7) := 7^ (7) = F 1 (7) (/ + 7/1), 
fo, fi G H, is holomorphic for |7| = r > Ri then (according to Lemma 1.3 in [31]) 



Jn[t,K 
In ||^ (7) || < ci lnr + C2 / : — —dt, 

'0 



t 



where n yt, KJ is the number of s- values of operator K on interval {t l , 00). Since 

A n (K) = 0(n~ l ), we have [9J A n (k\ = Ofa -1 ), n -* 00, and n (t,K\ = 0(t), 
t — > 00. The following inequality holds 

i2 



In k3 (7) < c 4 |7l 



(34) 



Choose 9 < I and i?i > T (1 + cot 0). Estimates (155j) and (151]) allow us to 
apply the Phragmen-Lindeloeff principle [36J according to which the boundedness 
of functions \\(p on the sides of angle A, I7I > Ri and the holomorphic property 
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of p (7) yield the boundedness of cp (7) for all I7I > R\ (including the points inside 
angle A); the following inequality holds 

;i + cot eHllfoWRi 1 + 

Ri 



||(^(7)|| <max jr- — — , max |M7)II . (35) 

1 — 4r (1 + cot 0) h\=Ri 



Thus vector-function ip (7) is uniformly bounded with respect to the norm in domain 
| 7 |>i2o- 

Theorem 4. In domain C\Iq the spectrum of pencil L (7) forms a countable 
set of isolated eigenvalues of finite algebraic multiplicity with an accumulation point 
at infinity. 

Proof. It is sufficient to prove that the spectrum of L (7) (or F (7)) is not empty 
in domain \ r y\> R for any R (see Theorem H]). 

Assume that the statement of the theorem is wrong. Then the vector-function 
^(7) = F^ 1 (7)/ is holomorphic in domain |7| > R for any / G H. Take R > 
sJ~S\ + £2 and assume that operator-function F (7) has a bounded inverse on circum- 
ference I7I = R. From estimate ( 135|) it follows that \\<p (7)!) < c ||/|| , I7I > R, V/ G 
H, and (7)!) < c, I7I > R. Let us integrate the equality 



(l 2 K + i) * f - F- 1 (7) / = (i 2 K + /) 1 —T (7) F- 1 (7) / 



on arbitrary contour which contains inside only one eigenvalue 7^ of operator- 
function + I (for example, Tk := {7 : |7 — 7fc| = o~k} for sufficiently small 
8k > 0). In this case the integral of F~ l (7) / is equal to zero and integrals of other 
terms are equal to the residuals at point 7^. Apply the expansion [2T] for resolvent 
R (7) in the vicinity of 7^ 



Uk + i) 1 = 1 ^p fc + 5fc ( 7 ) 



27fc 1 -Ik 

(operator-function S k (7) is holomorphic in the vicinity of 7^ and P k is an eigenpro- 
jector corresponding to eigenvalue 7*.). The expansion yields 

P k (l- 1 - 1 T( lk )F~ l ( lk ))f = 0. 

In line with estimates ||T (7) F^ 1 < T c, I7I > R, we obtain that for sufficiently 
large 7^ (such 7^ can be always chosen since eigenvalues of a compact operator 
K > have an accumulation point at zero) the operator I — 7^ T (7^) F _1 (7^) has 
a bounded inverse. Since / is arbitrary we have P k f = for all / G H . However 
Pfc 7^ 0, so that the latter is not possible. This contradiction proves the theorem. 
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Figure 3: Spectrum of pencil L (7) on the complex plane, o and x denote, respec- 
tively, the degeneration points and eigenvalues of pencil L (7) which are not equal to 



Figure [3] shows the distribution of the spectrum of pencil L (7) on the complex 
plane. 

5 Conclusion 

We have reduced the boundary eigenvalue problem for the Maxwell equations de- 
scribing normal waves in a broad class of nonhomogeneously filled waveguides to an 
eigenvalue problem for an operator pencil. We have proved fundamental properties 
of the spectrum of normal waves: the spectrum is nonempty and forms a countable 
set of isolated points on the complex plane (cut along two intervals on the real axis) 
without finite accumulation points, is localized symmetrically in a strip, and contains 
not more than a finite number of real points. 

The results obtained in this work are of fundamental character for the mathemat- 
ical theory of wave propagation in guides and must be used when particular types of 
nonhomogeneously filled waveguides are considered in various applications. 
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